{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 线性回归"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "吴恩达机器学习ex1的python实现  \n",
    "ex1data1.txt -单变量线性回归数据集  \n",
    "ex1data2.txt -多变量线性回归数据集 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 一、简单练习"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1., 0., 0., 0., 0.],\n",
       "       [0., 1., 0., 0., 0.],\n",
       "       [0., 0., 1., 0., 0.],\n",
       "       [0., 0., 0., 1., 0.],\n",
       "       [0., 0., 0., 0., 1.]])"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#导包\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "sns.set(context=\"notebook\", style=\"whitegrid\", palette=\"deep\")\n",
    "#输出一个5*5的对角矩阵\n",
    "A=np.eye(5)\n",
    "A"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***\n",
    "## 二、单变量的线性回归"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 问题背景"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "&ensp; &ensp; 在本部分的练习中，您将使用一个变量实现线性回归用来预测餐车利润。假设你是某公司的CEO餐厅特许经营，正在考虑在不同的城市开设新的出口。连锁店已经在不同的城市有卡车，你有数据来自城市的利润和人口。您可以使用这些数据来帮助您选择下一个扩展到哪个城市。ex1data1.txt文件包含线性回归问题lem的数据集。第一列是一个城市的人口，第二列是该城市一辆餐车的利润。利润为负值表示亏损。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "&ensp; &ensp; 在开始任何任务之前，通过可视化来理解数据通常是有用的。 对于这个数据集，您可以使用散点图来可视化数据，因为它只有两个属性需要绘制(利润和人口)。 (您将在现实生活中遇到的许多其他问题是多维的，无法在二维图上绘制出来。) 在例1，数据集从数据文件加载到变量X和y中。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "## header = None表示没有表头\n",
    "df=pd.read_csv('ex1data1.txt',header=None,names=['population','profit'])\n",
    "#df是pandas的DataFrame类型\n",
    "#取df列数（0表示行，1表示列）\n",
    "n=df.shape[1]\n",
    "#取人口列\n",
    "X=df.iloc[:,0:1]\n",
    "#取利润列\n",
    "y=df.iloc[:,1:2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>population</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>6.1101</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5.5277</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>8.5186</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>7.0032</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5.8598</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>92</th>\n",
       "      <td>5.8707</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>93</th>\n",
       "      <td>5.3054</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>94</th>\n",
       "      <td>8.2934</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>95</th>\n",
       "      <td>13.3940</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>96</th>\n",
       "      <td>5.4369</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>97 rows × 1 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "    population\n",
       "0       6.1101\n",
       "1       5.5277\n",
       "2       8.5186\n",
       "3       7.0032\n",
       "4       5.8598\n",
       "..         ...\n",
       "92      5.8707\n",
       "93      5.3054\n",
       "94      8.2934\n",
       "95     13.3940\n",
       "96      5.4369\n",
       "\n",
       "[97 rows x 1 columns]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***\n",
    "### 绘制散点图观察原始数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "*c* argument looks like a single numeric RGB or RGBA sequence, which should be avoided as value-mapping will have precedence in case its length matches with *x* & *y*.  Please use the *color* keyword-argument or provide a 2-D array with a single row if you intend to specify the same RGB or RGBA value for all points.\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAs8AAAHiCAYAAAAXqCHCAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAA4E0lEQVR4nO3dfXhU9Z3//9dkJhkc4i84vw3RNYjf9Wdkt7TGoFXrTVa9BBRSxJsURbxhrbSlRv3W+hWkF11rkaXs4jcrXd3Wm0ulKyMWEK5qy9Y2tkWtBoPGtcQqEREb0eDIEDK35/cHzZBJQnKSzJlzzszzcV29ypyZyXzyybnG17znfT4fj2EYhgAAAAAMqcjuAQAAAABuQXgGAAAATCI8AwAAACYRngEAAACTfHYPwKxUKqUDBw6ouLhYHo/H7uEAAAAgTxmGoXg8rrFjx6qoKLPW7JrwfODAAbW1tdk9DAAAABSIqqoqHX300RnHXBOei4uLJR36JUpKSnL2uq2trZo8eXLOXq8QMcfWY46txfxajzm2FvNrPebYetmc41gspra2tnT+7M014bmnVaOkpER+vz+nr53r1ytEzLH1mGNrMb/WY46txfxajzm2XrbneKBWYS4YBAAAAEyytPL8wAMP6LnnnpMk1dbW6s4779SiRYvU3Nyso446SpL07W9/WxdffLGVwwAAAACywrLwvHXrVv3+97/X+vXr5fF4dNNNN2nLli1qbW3Vk08+qfHjx1v10gAAAIAlLGvbKC8v11133aWSkhIVFxfrpJNO0p49e7Rnzx4tXrxYdXV1amxsVCqVsmoIAAAAQFZZFp5PPvlkVVdXS5La29v13HPP6bzzztNZZ52lZcuWKRQK6bXXXtO6deusGgIAAACQVR7DMAwrX+Cdd97RggULdMstt2j27NkZ923ZskUbNmzQ6tWrh/w50WhUra2tVg0TAAAAyDB58uR+K3hYesFgc3OzGhoatHjxYs2YMUM7duxQe3u7pk2bJunQ7i0+3/CGMNAvYaXm5mZNmTIlZ69XiJhj6zHH1mJ+rcccW4v5tR5zbL1szvFgRVvL2jY++ugjLVy4UCtXrtSMGTMkHQrLy5YtUzgcVjwe19q1a1lpAwAAAK5hWeX54YcfVjQa1fLly9PH5syZo5tvvllXX321EomEpk6dqpkzZ1o1BAAAACCrLAvPS5Ys0ZIlSwa8b+7cuVa9LAAAAGAZdhgEAAAATCI8AwAAACYRngEAAACTCM8AAACASYRnAAAAOE44ElXbrn0KR6J2DyWDpZukAAAAAMPVtG23GkMt8nk9SiQNNdRXq7am0u5hSaLyDAAAAAcJR6JqDLUoFk+qqzuhWDypxlCLYyrQhGcAAAA4Rkdnl3xeT8Yxn9ejjs4um0aUifAMAAAAx6gIBpRIGhnHEklDFcGATSPKRHgGAACAY5SV+tVQX62SYq8CY3wqKfaqob5aZaV+u4cmiQsGAQAA4DC1NZWqripXR2eXKoIBxwRnifAMAAAAByor9TsqNPegbQMAAAAwifAMAAAAmER4BgAAAEwiPAMAAAAmEZ4BAAAAkwjPAAAAgEmEZwAAAMAkwjMAAABgEuEZAAAAMInwDAAAAJhEeAYAAABMIjwDAAAAJhGeAQAAAJMIzwAAAIBJhGcAAJA3wpGo2nbtUzgStXsoyFM+uwcAAACQDU3bdqsx1CKf16NE0lBDfbVqayrtHhbyDJVnAADgeuFIVI2hFsXiSXV1JxSLJ9UYaqECjawjPAMAANfr6OySz+vJOObzetTR2WXTiJCvCM8AAMD1KoIBJZJGxrFE0lBFMGDTiJCvCM8AAMD1ykr9aqivVkmxV4ExPpUUe9VQX62yUr/dQ0Oe4YJBAACQF2prKlVdVa6Ozi5VBAMEZ1iC8AwAAPJGWamf0AxL0bYBAAAAmER4BgAAAEwiPAMAAAAmEZ4BAAAAkwjPAAAAgEmEZwAAAAcIR6Jq27WPLcUdjqXqAAAAbNa0bbcaQy3yeT1KJA011FertqbS7mFhAFSeAQAAbBSORNUYalEsnlRXd0KxeFKNoRYq0A5FeAYAALBRR2eXfF5PxjGf16OOzi6bRoTBEJ4BAABsVBEMKJE0Mo4lkoYqggGbRoTBEJ4BAABsVFbqV0N9tUqKvQqM8amk2KuG+mq2GXcoLhgEAACwWW1NpaqrytXR2aWKYIDg7GCEZwAAAAcoK/UTml2Atg0AAADAJMIzAAAAYBLhGQAAADCJ8AwAAACYRHgGAABwkHAkqrZd+9hh0KFYbQMAAMAhmrbtVmOoRT6vR4mkoYb6atXWVNo9LPRC5RkAAMABwpGoGkMtisWT6upOKBZPqjHUQgXaYQjPAAAADtDR2SWf15NxzOf1qKOzy6YRYSCEZwAAAAeoCAaUSBoZxxJJQxXBgE0jwkAIzwAAAA5QVupXQ321Soq9CozxqaTYq4b6anYddBguGAQAAHCI2ppKVVeVq6OzSxXBAMHZgQjPAAAADlJW6ic0OxhtGwAAAIBJhGcAAADAJMIzAAAAYBLhGQAAADCJ8AwAAACYRHgGAAAATLI0PD/wwAOaMWOGZsyYoRUrVkiStm7dqrq6Ok2dOlWrVq2y8uUBAACArLIsPG/dulW///3vtX79em3YsEFvvfWWNm/erMWLF+vHP/6xfvGLX6i1tVVNTU1WDQEAAADIKsvCc3l5ue666y6VlJSouLhYJ510ktrb2zVx4kRNmDBBPp9PdXV1ev75560aAgAAAJBVHsMwDKtfpL29XVdffbWuvfZa7dy5UytXrpR0qDr905/+VI888siQPyMajaq1tdXqoQIAAACSpMmTJ8vvz9zt0fLtud955x0tWLBAd955p7xer9rb29P3GYYhj8czrJ830C9hpebmZk2ZMiVnr1eImGPrMcfWYn6txxxbi/m1HnNsvWzO8WBFW0svGGxubtYNN9yg73znO5o9e7aOPfZY7d27N33/3r17NX78eCuHAAAAAGSNZeH5o48+0sKFC7Vy5UrNmDFDknTqqadq586dev/995VMJrV582adf/75Vg0BAAAAyCrL2jYefvhhRaNRLV++PH1szpw5Wr58uW655RZFo1HV1tZq+vTpVg0BAAAAyCrLwvOSJUu0ZMmSAe979tlnrXpZAAAAwDLsMAgAAACYRHgGAAAATCI8AwAAACYRngEAAACTCM8AAACASYRnAAAAwCTCMwAAAGAS4RkAAAAwifAMAAAAmER4BgAAAEwiPAMAAAAmEZ4BAAAAkwjPAAAAgEmEZwAAAMAkwjMAAABgEuEZAAAAMInwDAAAAJhEeAYAAABMIjwDAAAAJhGeAQAAAJMIzwAAAIBJhGcAAADAJMIzAAAAMoQjUbXt2qdwJGr3UBzHZ/cAAAAA4BxN23arMdQin9ejRNJQQ321amsq7R6WY1B5BgAAgKRDFefGUIti8aS6uhOKxZNqDLVQge6F8AwAAABJUkdnl3xeT8Yxn9ejjs4um0bkPIRnAAAASJIqggElkkbGsUTSUEUwYNOInIfw7GI08wMAgGwqK/Wrob5aJcVeBcb4VFLsVUN9tcpK/XYPzTG4YNClaOYHAABWqK2pVHVVuTo6u1QRDBCc+yA8u1DvZv5Y/NCxxlCLqqvKOcEBAMColZX6yRRHQNuGC9HMDwAAYA/CswvRzA8AAGAPwrML0cwPAABgD3qeXYpmfgAAgNwjPLsYzfwAAAC5RdsGAADIG+yBAKtReQYAAHmBPRCQC1SeAQCA6/XeA6GrO6FYPKnGUAsVaGQd4RkAALgeeyAgVwjPAADA9dgDAblCeAYAAK7HHgjIFS4YBAAAeYE9EJALhGcAAJA32AMBVqNtAwAAADCJ8AwAAACYRHgGAAAATCI8AwAAACYRngEAAACTCM8AAACASYRnAAAAwCTCMwAAAGAS4RkAAAAwifAMAAAAmER4BgAAAEwiPAMAAAAmEZ4BAAAAkwjPAAAAgEmEZwCAKeFIVG279ikcido9FACwjc/uAQAAnK9p2241hlrk83qUSBpqqK9WbU2l3cMCgJyj8gwAGFQ4ElVjqEWxeFJd3QnF4kk1hlqoQAMoSIRnAMCgOjq75PN6Mo75vB51dHbZNCIAsA/hGQAwqIpgQImkkXEskTRUEQzYNCIAsA/hGQAwqLJSvxrqq1VS7FVgjE8lxV411FerrNRv99AAIOe4YBAAMKTamkpVV5Wro7NLFcEAwRlAwSI8AwBMKSv1E5oBFDxL2zYikYhmzpyp3bt3S5IWLVqkqVOnatasWZo1a5a2bNli5csDAAAAWWVZ5Xn79u1asmSJ2tvb08daW1v15JNPavz48Va9LAAAAGAZyyrPoVBIS5cuTQflgwcPas+ePVq8eLHq6urU2NioVCpl1csDAAAAWecxDMMY+mEjd+GFF+rxxx+XYRhavny5li5dqqOPPloLFizQzJkzVV9fb+rnRKNRtba2WjlUAAAAIG3y5Mny+zOv9cjZBYMTJkzQ6tWr07fnzZunDRs2mA7PPQb6JazU3NysKVOm5Oz1ChFzbD3m2FpWzm84EmWFC3EOW435tR5zbL1szvFgRduchecdO3aovb1d06ZNkyQZhiGfj8U+AOBImrbtVmOoRT6vR4mkoYb6atXWVNo9LAAoaDnbJMUwDC1btkzhcFjxeFxr167VxRdfnKuXBwBXCUeiagy1KBZPqqs7oVg8qcZQi8KRqN1DA4CClrPS76RJk3TzzTfr6quvViKR0NSpUzVz5sxcvTwAuEpHZ5d8Xo9i8cPHfF6POjq7Crp9AwDsZnl4fuGFF9L/njt3rubOnWv1SwKA61UEA0okM6/nTiQNVQQDNo0IACDlsG0DAGBeWalfDfXVKin2KjDGp5Jirxrqq6k6A4DNuGIPAByqtqZS1VXlrLYBAA5CeAYABysr9ROaAcBBaNsAAAAATCI8AwAAACYRngEAAACTCM8AAACASYRnAAAAwCTCMwAAAGAS4RkAAAAwifAMAAAAmER4BnoJR6Jq27VP4UjU7qEAAAAHYodB4K+atu1WY6hFPq9HiaShhvpq1dZU2j0sAADgIFSeAR2qODeGWhSLJ9XVnVAsnlRjqIUKNLKObzcAwN2oPAOSOjq75PN6FIsfPubzetTR2aWyUr99A0Ne4dsNAHA/Ks+ApIpgQImkkXEskTRUEQzYNCLkG77dAID8QHgGJJWV+tVQX62SYq8CY3wqKfaqob6aqjOypufbjd56vt0AALgHbRvAX9XWVKq6qlwdnV2qCAYIzsgqvt0AgPxA5RnopazUr6oTjiE4I+v4dgMA8gOVZwDIEb7dAAD3IzwDQA6VlfoJzQDgYrRtAAAAACYRngEAAACTCM8AAACASYRnAAAAwCTCMwAAAGAS4RkAAAAwifAMAAAAmER4BgAAAEwiPAMAAAAmEZ4BAAAAkwjPAAAAgEmEZwAAAMAkwjNgg3AkqrZd+xSORO0eCgAAGAaf3QMACk3Ttt1qDLXI5/UokTTUUF+t2ppKu4cFOEo4ElVHZ5cqggGVlfrtHg4ApBGegQFY9R/ucCSqxlCLYvGkYvFDxxpDLaquKicgAH810AfMUrsHBQB/RXgG+rCyMtzR2SWf15MOzpLk83rU0dlFeAZ05A+Yt9aNt3dgAPBX9DxjRPKhZ3eg36H3f7i7uhOKxZNqDLVk7fesCAaUSBoZxxJJQxXBQFZ+PuB2PR8we/N5PfrsQNKmEQFAJirPGLZ86Nk90u9gdWW4rNSvhvrqfq9N1Rk45EgfMMeN9do0IgDIRHjGsORDz+5gv0MuKsO1NZWqrirnYihgAEf6gDnW6LB7aAAgifCMYcqHnt3BfoeqE47JSWW4rNTvmvkCcm2gD5jNzYRnAM5AeMaw5EPP7lC/A5VhwH58wATgVFwwiGHp+Uq1pNirwBifSoq9ruvZNfM7lJX6VXXCMa76vQAAgPWoPGPY8qEymw+/AwAAyD3CM0YkH75SzYffAQAA5BZtGwAAAIBJpsLzf//3f/c7tmHDhmyPBQAAAHC0Qds2XnjhBSUSCa1YsUKpVCp9PJFI6N///d912WWXWT0+AAAAwDEGDc9vv/22Xn75ZX366ad64oknDj/J59MNN9xg9dgAAAAARxk0PC9cuFALFy7UmjVrNHfu3FyNCQAAAHCkQcPzxo0bNWvWLEWjUT366KP97r/xxhstGxgAAADgNIOG5127dkmS3nnnnZwMBgAAAHCyQcPz9u3bJUmTJk3S9ddfn5MBAQAAAE41aHh+9913tWnTJj3xxBP627/9WxmGkXH/1KlTLR0cAAAA4CSDhueGhgatW7dOn376qR5//PGM+zweD+EZAAAABWXQ8Dx79mzNnj1b9913nxYtWpSrMQEAAACONGh47nHnnXfqJz/5iV588UUlEgmdc845+sY3viGfz9TTAQAAgLxganvuVatW6eWXX9b111+vG2+8Ua+//rpWrFhh9dgAAAAARzFVOn7xxRf1zDPPqLi4WJL0j//4j/rqV7+qxYsXWzo4AAAAwElMVZ4Nw0gHZ0kqKSnJuA0AAAAUAlPhedKkSVq2bJl27dqlDz74QPfdd5+qqqqsHhsAAADgKKbC89KlS/X5559rzpw5qq+vV2dnp773ve9ZPTYAAADAUUz1PD/00ENavny51WMBAAAAHM1U5fm3v/2txcMAgMIQjkTVtmufwpGo3UMBAIyAqcpzZWWl5s+fr5qaGo0dOzZ9/MYbbxz0eZFIRHPmzNGDDz6oyspKbd26Vffdd5+i0aguueQS3X777aMbPQC4SNO23WoMtcjn9SiRNNRQX63amkq7hwUAGAZT4XncuHGSpLfeekter1dHH330kM/Zvn27lixZovb2dklSd3e3Fi9erCeeeELHHXecFixYoKamJtXW1o548ADgFuFIVI2hFsXiScXih441hlpUXVWuslK/vYMDAJhmqm3jpptuUltbm5qamvTCCy/ogw8+0C233DLoc0KhkJYuXarx48dLkt544w1NnDhREyZMkM/nU11dnZ5//vnR/wYA4AIdnV3yeT0Zx3xejzo6u2waEQBgJEyF58WLF6u+vl7bt29XS0uLpk2bprvvvnvQ5/zwhz/U6aefnr798ccfq7y8PH17/Pjx6ujoGOGwAcBdKoIBJZJGxrFE0lBFMGDTiAAAI2GqbePgwYP62te+lr49b948hUKhYb1QKpWSx3O46mIYRsZts1pbW4f9nNFqbm7O+WsWGubYesyxtczMb90Z/482vvKZvEVSMnXo9p935P49za04h63F/FqPObZeLubYVHj+u7/7O23btk01NTWSpLa2NlVWDu8il2OPPVZ79+5N3967d2+6pWM4Jk+eLL8/d/2Bzc3NmjJlSs5erxAxx9Zjjq1ldn6nTJFmT4uqo7NLFcEAvc7DwDlsLebXesyx9bI5x9Fo9IgFW1Phec+ePZo3b55OOeUU+Xw+/c///I/Ky8tVV1cnSdq0adOQP+PUU0/Vzp079f7776uyslKbN2/WFVdcMYxfAwDcr6zUT2gGABczFZ7vuOOOUb+Q3+/X8uXLdcsttygajaq2tlbTp08f9c8FAAAAcsVUeP7yl7884hd44YUX0v8+++yz9eyzz474ZwEAAAB2MrXaBgAAAADCMwAAAGAa4RkAAAAwifAMAAAAmER4zpFwJKq2XfsUjkTtHgoAAABGyNRqGxidpm271Rhqkc/rUSJpqKG+WrU1w9tkBgAAAPaj8myxcCSqxlCLYvGkuroTisWTagy1UIEGkBV8qwUAuUXl2WIdnV3yeT2KxQ8f83k96ujsYpcxAKPCt1oAkHtUni1WEQwokTQyjiWShiqCAZtGBCAf8K0WANiD8GyxslK/GuqrVVLsVWCMTyXFXjXUV1N1BjAqPd9q9dbzrRYAwDq0beRAbU2lqqvK1dHZpYpggOAMYNT4VgsA7EHlOUfKSv2qOuEYgjOArOBbLQCwB5VnAHApvtUCgNwjPAOAi5WV+gnNAJBDtG0AAAAAJhGeAQAAAJMIzwAAAIBJhGcAAADAJMIzAAAAYBLhGQAAADCJ8AwAAACYRHgGkFfCkajadu1TOBK1eygAgDzEJikA8kbTtt1qDLXI5/UokTTUUF+t2ppKu4cFAMgjVJ4B5IVwJKrGUIti8aS6uhOKxZNqDLVQgQYAZBXhGShw+dLm0NHZJZ/Xk3HM5/Woo7PLphEBAPIRbRtAAcunNoeKYECJpJFxLJE0VBEM2DQiAEA+ovIMFKh8a3MoK/Wrob5aJcVeBcb4VFLsVUN9tcpK/XYPDQCQR6g8AwWqp80hFj98rKfNwa2Bs7amUtVV5ero7FJFMODa3wMA4FyEZ8DFwpHoiINivrY5lJX6Cc0AAMsQngGXGm2/ck+bQ9+fQfAEAODICM+AC/XuV+5pu2gMtai6qnxY4Zc2BwAAhofwDLhQNvuVaXMAAMA8VttAQXPrGsf52q8MAIDTUXl2qNFcCAZz3LzGMf3KAADYg/DsQG4OdW6RrZ5hO9GvDABA7tG24TD5tnGFU+XLVs5lpX5VnXCMK4KzW1tkAADojcqzw+TjxhVORM9wbvFtCgAgX1B5dhhCXW6wlXPu8G0KACCfUHl2mPSFYGtfV1GRR6kUF4JZhZ7h3ODbFABAPiE8O5TR63+wTq7XOHbqKipWjotvUwAA+YTw7DA9X3HHE6n0MbetAjESZsObU8OnGU7t+7V6XCyrBwDIJ4RnhynEr7jfbD+gZU9vGTK8OTV8muG0pfF6PoQc5fflZFy0yAAA8gXh2WGs/orbaZXbcCSqja98pkTSGDS8OS18DpeTPhT1/hASi6dU1OeyYavGxTbgAIB8QHh2GCu/4nZi5bajs0veIimRPHxsoPA2mvDphA8MTun7HehDiJKZj6EfGQCAIyM8O5AVX3E7tXJbEQwomco8NlB4G2n4dMoHBqf0/Q70IaTY55HkUbGviH5kAACGQHh2qGx/xe2ktoHeykr9mnXmOG169fNBQ+VIwqfTPjA4oe93oA8hHk+R7r+9VgejCce08wAA4FSE5wLhlLaBgXzxxLGaPe3MIUPlcMOnEz8w2N33e6QPIRMqjrZtTAAAuAnhuUA4pW3gSMyGyuGETyd/YLCTEyrgAAC4FeG5gBRaaHL6BwY72V0BBwDArQjPBabQQlOhfWAAAADWIjwj7xXaBwYAAGCdoqEfgnwWjkTVtmufwpGo3UMBAABwPCrPBcwpayADAAC4BZXnAtV7DeSu7oRi8aQaQy1UoAEAAAZBeC5QPWsg99azBnI+o00FAACMBm0bJoQj0bxbraEQ10CmTQUAAIwW4XkIb7Yf0LKnt+Rd4Cq0NZCdtlU3AABwJ8LzIMKRqDa+8pkSSSMvA1chrYHsxK26AQCA+xCeB9HR2SVvkZRIHj7mpsBlpt2kUNZALsQ2FQAAkH2E50FUBANKpjKPuSVw0d+byU1tKvnYYw8AQL4gPA+irNSvWWeO06ZXP3d84OptqP7ebIczt4Q9N7Sp8KFn9NxyPgIA3InwPIQvnjhWs6ed6ar/GA/U3+stOtRu0tK2N6vhzG1hz8ltKlzUOHpuOx8BAO7DOs8mlJX6VXXCMa4JMAP19x6MJtT67idZ3RiFjVayq1DX3s4WzkcAQC4Qnk1y0+YaZaV+zZ1+Sr/jTzz3J3mLshfOCj3sZfuc4KLG0Sn08xEAkBu0bZjgtq+Cm7bt1pPP/anf8WKfR4k+V0COJpwVctiz4pxw00WNTlTI5yMAIHdsCc/z5s1TZ2enfL5DL3/PPffo1FNPtWMoQzrQnVTjJvf0ofZ8dR1PpPrdl0xJX581WT999q2shDMrwp4bLvaysjfZDRc1OhUfPgAAuZDz8GwYhtrb2/Wb3/wmHZ6d7LMDSVdtrjHQxYKS5PMWpaujX/nS32YtnGUz7Lmlwm/1hitOvqjR6fjwAQCwWs7T63vvvSdJmj9/vj777DPV19fr2muvzfUwTBs31uuqr4IH+uq62Fek//u//1ETKo6WlP1wNtDPG24F2U0rTdAe4Gx8+AAAWMljGIYx9MOy5/XXX9d//dd/6Xvf+57i8biuu+46LVq0SOecc86gz4tGo2ptbc3RKDO92X5AG1/5TN6iQ60Ps84cpy+eONaWsZhh93hH8voffhrT4y/sVTR++HT0F3t03YXlOv7/LbF6yMNm9xwDAADrTZ48WX5/ZkEm5+G5r8cee0x79uzR4sWLB31cT3ge6JewUnNzs6ZMmeKKXtze7BpvOBLV/Hu3KBY/vKd5SbFXjyy5+IjjaG5u1v93yuRhP89ubjones5jWIP5tR5zbC3m13rMsfWyOceD5c6cL1X32muv6aWXXkrfNgzDFb3Pblvr2a7xjnS5sJ6LvUqKvQqM8amk2Ov4i73cdk4AAIDRy3lq3b9/vxobG/XUU08pHo9r/fr1+ud//udcD8N2bqpaDsdo+oG52AsAADhdzsPzBRdcoO3bt+uyyy5TKpXSNddco9NOOy3Xw7CVW1aVGInRLhc2nIu98vUDCAAAcC5b+iVuu+023XbbbXa8tO3ctKrESOWigpzPH0AAAIBzsT13jhXKFsJW9gP3/gDS1Z1QLJ5UY6jFFVunAwAAdyM85xhrBB8WjkTVtmufDnQnh35wL4XyAQQAADiP85e5yAN9e3MLfQvhcCSq515qV+i/21TsK1IsnpRx1O5028VQvcxWfgChjxoAAAyG8GyxI/XmFuqqEk3bdqtx7euKJVKSpPhf/7+n77ulbe+QvcxWfQChjxoAAAyF8GyhoS4OLKTQLPWaj78G5t58Xo/e3f2Z6YspB/sAMpLqcSFcyAkAAEaP8Gyhnt7cnjAmHe7NLcRANtB89EgkDcnjGdZ8DfQBZKTVY/5WAADADC4YtBAXBx4WjkQVORhPt2n05i2SGuqrddLxZaOar9GswsHfCgAAmEF4tpAbt5w+kp6VMUayHFzTtt2af+8W/cvjr8owDHmLPIfmw1ekudMn6X9fdpxqaypHPV+jWYXDrr/VaOYVAADkHm0bFsuHiwNHcyHdQL3EJb4i/Z95p+ukynEqK/XrxT/8UW279qkiGBjVfI22epzrvxUXKAIA4D6EZxNGu3yZmy8OHO2FdAP2EvuKVBooUVmpX03bduv+jX9RSfHejAA50nke7SocufpbcYEiAADuRHgewpvtB7Ts6S0FWx0c7YV0g1WDewJkImkokUxIGn2AdEulnwsUAQBwJ3qeBxGORLXxlc/yYhvokfbWjrYVYrBeYqt2CrRya/Bs4QJFAADcicrzIDo6u+QtkhK9do92Y3VwNL212WiFOFI1uJADJDtNAgDgToTnQVQEA0r2WVnNTLhz0hbP2eitzUYrxEC9xD0B8v6ntqmk2FtwAdItLSYAAOAwwvMgykr9mnXmOG169XPT1UGnraCQrd5aqy6kq62plOfgHh074eSCDJBuvpgUAIBCRHgewhdPHKvZ0840VR00W+XNZWXaDa0RY8d4VXXCMXYPAwAAYEiEZxPMVgfNVHlzXZm2q7fWSa0rAAAA2UJ4zqKhqrxHqkz/3fFlOhhNWBY02fwDAAAgOwjPWTRUlXegyrRk6JZ//Y183iKlUoZu/dpplgRNNv8AAAAYPcJzlg1W5R2oMh2LH1rOI5k8tB7eqv/aNqKg6ZQ2CTb/AAAA+YzwbIEjVXnLSv266atf0E82tsrnLVI8kVKiz1p4yZShd3d/pppJFaZfz0ltEm64QBEAAGCk2GEwh5q27dZPn31LPq9H8URK08+eOPADPZ6Bjw+gd5uEE3ZBHGxHQQAAALej8pwjvUNuj1+9/L68RcrYiMXn9eik48tM/1wntkmw+QcAAMhXVJ5NCkeiatu1b8QV3Z6Q25vPV6Q5UyepxFekMSVelfiKdNucmmGFTae2SZSV+lV1wjEEZwAAkFeoPJuQjZ7iI4XcS84+UZecfeKIq7R2reMMAABQiAjPQzjQnVTjptEvvTZUyB1N2KVNAgAAIDcIz0P47EAyaz3FVobcXK3jDAAAUMgIz0MYN9ab1Z5iQi4AAIB7ccHgEMaO8bL0GgAAACRReTYlVz3F2dol0Cm7DQIAAOQbwrNJVrdbZGuXQCftNggAAJBvaNtwgGztEui03QYBAADyDeHZAQbcQOWvK3rY8XMAAAAwMMKzA2Rrl0Cn7jYIAACQLwjPDtCzgcpoV/TI1s8BAADAwLhgcBisXMUiWyt6sNsgAACAdQjPJg21ikU2gnW2VvRgIxYAAABrEJ5N6L2KRc823Y2hFlVXlaus1M/ycAAAAAWC8GxCzyoWPcFZylzFYrBgbUbvqnXP6/X9N5VkAAAA+xGeTRhsFYtDAdro8wxDHZ1dpgJv76p1LJ6UYUj+Em/Gv6lmAwAAOAOrbZgw2CoWR/l9isVTGY+PxVM6yj/055K+m5okkoaSKaPfv9nsBAAAwBmoPJt0pFUsDkYTKvZ5FE8crj4X+zw6GE0M+TMHagc5kp42Edo3AAAA7EN4HoaBVrGoCAbk8RRJSqaPeTxFpjYmGagd5EjY7AQAAMB+tG2M0mg2Jun7XJ/XI2+Rp9+/2ewEAADAGag8Z8FoNibp+1wpe6ttWLmpCwAAQCEiPGfJaDYm6fvcI/17OFh7GgAAIPto23CRcCSqtl37hlx1o+8qHqzWAQAAkB1Unl1iOJXkwTZ1oX0DAABg5Kg8u8BwK8mDbeoCAACAkSM8u0BPJbm33tuD9zWaFUAAAABwZLRt5EDPqhdH+X06GE0Me/WLkVSSR7MCyEiwsgcAACgEhGeL9fQqS4Zi8ZSKfR55PEXDWv2ip5L8f9e+rqIij1Ipw1QleTQrgAwHK3sAAIBCQduGhXr3KsfiKUlSPGGMePULT6//OQUrewAAgEJCeLbQQL3KPQbrWe4rHVATKXXHkoolUo4JqMPtxwYAAHAzwrOFBupV7pFIpBTpipkKwE4OqKzsAQAACgnh2UK9V70oKT401cU+j3xej5IpQ//yxGuaf+8WNW3bPejPGU1ANbuxykixsgcAACgkXDBooXAkquP+Zqzuv71WB6MJHeX3ae9nB3XvI68okUypqzshSWoMtai6qvyIgbMnoPa9KG+ogJqrC/lyvbIHAACAXQjPw2R2Sbbntu7UTza2yuctUjJ1OLgejCZU7CtSPJFKP9bM7n/DDaiZFyseOjZUSB+NXK3sAQAAYCfC8zCYreQ+t3WnfvzMG5KUDsk9wXWgFox4IqXIwbjCkeigAXQ4AZUtugEAALKPnmeTzC7JFo5E9Z8bWvs931t0OLj27hH2FnlkGIb+5fFXTfU/m8WFfAAAANlHeDbJ7IoXHZ1dKvb1X54ukUylg2ttTaUeWXKx/s+80+UtOlTFzvYayVzIBwAAkH20bZhktpJbEQwomVI/X581OSO4lpX6VRookc9XpNgw+5/NGqxPmu20AQAAho/wbFLfFS/iiZSuuujkQR/nLTq0o+DNl03WJV/5X/0em4vWioH6pNlOGwAAYGQIz8PQU8l9bmu7nv51m9b/9s96+tfv9AufA1V8B6r0jnQJutHI9SocTkPFHQAAjAbheQSefuEdxRKpdLvFQOGzd8V3sEpvrtdILuRVOKi4AwCA0bLlgsFNmzbp0ksv1dSpU7VmzRo7hjBiw90q28wqHWWlflWdcExOwmuhrsJhdrUUAACAweQ8PHd0dGjVqlX62c9+pg0bNmjt2rX685//nOthjNhww+dww7bVCnUVDqf9HQAAgDvlvG1j69atOuusszRu3DhJ0rRp0/T888/r29/+dq6HMiLD7VN2YqW3ELfTduLfAQAAuI/HMAxj6Idlz0MPPaSuri7dfvvtkqSnn35ab7zxhn7wgx8M+rxoNKrW1v6bj9jlQHdSnx1IatxYr8aO8Q762DfbD2jjK5/JWyQlU9KsM8fpiyeOzdFI0YO/AwAAGI7JkyfL788sMua88pxKpeTxHP763DCMjNtDGeiXsFJzc7OmTJkyqp8xZYo0exqrPBxJNubYjEL+O+RqjgsV82s95thazK/1mGPrZXOOByva5jw8H3vssXrttdfSt/fu3avx48fnehg5N9B6y8g9/g4AAGA0cn7B4Fe+8hW99NJL6uzs1MGDB/WrX/1K559/fq6HAQAAAAxbzivPFRUVuv3223XdddcpHo/ryiuv1Je+9KVcDwMAAAAYNls2Samrq1NdXZ0dLw0AAACMmC2bpAAAAABuRHgGAAAATCI8j1A4ElXbrn1s7wwAAFBAbOl5drumbbv77TBYW1Np97AAAABgMSrPwxSORNUYalEsnlRXd0KxeFKNoRYq0AAAAAWA8DxMHZ1d8nkzd0T0eT3q6OyyaUQAAADIFcLzMFUEA0okjYxjiaShimDAphEBAAAgVwjPw1RW6ldDfbVKir0KjPGppNirhvpqtnwGAAAoAFwwOAK1NZWqripXR2eXKoIBgjMAAECBoPI8QmWlflWdcIwksWQdAABAgaDyPAosWQcAAFBYqDyPEEvWAQAAFB7C8wi5Zck6dkIEAADIHto2RsgNS9bRVgIAAJBdVJ5HyOlL1tFWAgAAkH1UnkfByUvW9bSVxOKHj/W0lThpnAAAAG5CeB6lslK/I8OoG9pKAAAA3Ia2jTzl9LYSAAAAN6LynMec3FYCAADgRoTnPOfUthIAAAA3om0DAAAAMInwDAAAAJhEeAYAAABMIjwDAAAAJhGeAQAAAJMIzwAAAIBJhGcAAADAJMIzAAAAYBLhGQAAADCJ8AwAAACYRHgewoHupNp27VM4ErV7KAAAALCZz+4BOFnTtt26f+NfVFK8V4mkoYb6atXWVNo9LAAAANiEyvMRhCNRNYZalEga6upOKBZPqjHUQgUaAACggBGej6Cjs0s+ryfjmM/rUUdnl00jAgAAgN0Iz0dQEQwokTQyjiWShiqCAZtGBAAAALsRno+grNSvhvpq+bweBcb4VFLsVUN9tcpK/XYPDQAAADbhgsFB1NZUynNwj46dcLIqggGCMwAAQIEjPA9h7Bivqk44xu5hAAAAwAFo2wAAAABMIjwDAAAAJhGeAQAAAJMIzwAAAIBJhGcAAADAJMIzAAAAYBLhGQAAADCJ8AwAAACYRHgGAAAATCI8AwAAACYRngEAAACTCM8AAACASYRnAAAAwCTCMwAAAGCSz+4BmGUYhiQpFovl/LWj0WjOX7PQMMfWY46txfxajzm2FvNrPebYetma45682ZM/e/MYAx11oP3796utrc3uYQAAAKBAVFVV6eijj8445prwnEqldODAARUXF8vj8dg9HAAAAOQpwzAUj8c1duxYFRVldjm7JjwDAAAAduOCQQAAAMAkwjMAAABgEuEZAAAAMInwDAAAAJhEeAYAAABMIjwDAAAAJhGeAQAAAJMIzwAAAIBJhOe/mjdvnmbMmKFZs2Zp1qxZ2r59e8b9b7/9ti6//HJNmzZNd999txKJhE0jdaenn346PbezZs3SlClTdM8992Q85oEHHtAFF1yQfsyaNWtsGq27RCIRzZw5U7t375Ykbd26VXV1dZo6dapWrVo14HP27NmjuXPnavr06frmN7+pAwcO5HLIrtN3jteuXauZM2eqrq5OixYtUiwW6/ec9evX69xzz02fz0f6W6D//C5atEhTp05Nz92WLVv6PYdzeHh6z3FTU1PG+/FZZ52lBQsW9HsO57B5DzzwgGbMmKEZM2ZoxYoVkngvzqaB5tfW92EDRiqVMs4991wjHo8f8TEzZswwXn/9dcMwDGPRokXGmjVrcjS6/NPW1mZcfPHFxqeffppxfMGCBca2bdtsGpU7tbS0GDNnzjS+8IUvGB988IFx8OBBo7a21ti1a5cRj8eN+fPnG7/97W/7Pe/mm282Nm/ebBiGYTzwwAPGihUrcj101+g7x++9955x8cUXG/v37zdSqZRx5513Go8++mi/591zzz3Gpk2bcj9gl+k7v4ZhGDNnzjQ6OjoGfR7nsHkDzXGPjz/+2LjooouMnTt39nse57A5f/jDH4yvfe1rRjQaNWKxmHHdddcZmzZt4r04Swaa34ceesjW92Eqz5Lee+89SdL8+fP11a9+VU8++WTG/R9++KG6u7tVXV0tSbr88sv1/PPP53qYeeP73/++br/9dgWDwYzjra2teuihh1RXV6d77rlH0WjUphG6RygU0tKlSzV+/HhJ0htvvKGJEydqwoQJ8vl8qqur63euxuNxvfrqq5o2bZokzueh9J3jkpISLV26VKWlpfJ4PKqqqtKePXv6Pe/NN9/U+vXrVVdXpzvuuEPhcDjXQ3eFvvN78OBB7dmzR4sXL1ZdXZ0aGxuVSqUynsM5PDx957i3FStWaM6cOTrxxBP73cc5bE55ebnuuusulZSUqLi4WCeddJLa29t5L86SgeY3FovZ+j5MeJb0+eef6+yzz9bq1av12GOP6amnntIf/vCH9P0ff/yxysvL07fLy8vV0dFhx1Bdb+vWreru7tYll1yScfzAgQP6+7//e333u9/V+vXr9fnnn+vHP/6xTaN0jx/+8Ic6/fTT07f7nqvjx4/vd67u27dPpaWl8vl8kjifh9J3jo8//nidc845kqTOzk6tWbNGF110Ub/nlZeX61vf+paeffZZHXfccf3alHBI3/n95JNPdNZZZ2nZsmUKhUJ67bXXtG7duozncA4PT9857tHe3q4//vGPuu666wZ8HuewOSeffHK6uNbe3q7nnntOHo+H9+IsGWh+Z86caev7MOFZ0mmnnaYVK1bo6KOPVjAY1JVXXqmmpqb0/alUSh6PJ33bMIyM2zDvqaee0o033tjv+NixY/WTn/xEJ510knw+n+bPn5/xN4A5Zs7VgY5xPg9fR0eHrr/+el1xxRU688wz+92/evVqTZkyRR6PRzfddJN+97vf2TBK95kwYYJWr16t8ePH66ijjtK8efP6vRdwDmfH2rVrdc0116ikpGTA+zmHh+edd97R/Pnzdeedd2rChAm8F2dZ7/nt+abErvdhwrOk1157TS+99FL6tmEY6U+CknTsscdq79696duffPLJgF9/YXCxWEyvvvqqLrzwwn737dmzJ6O61PdvAHP6nqt79+7td64Gg0Ht379fyWTyiI/B4N59913NmTNHs2fP1sKFC/vdv3//fj322GPp24ZhyOv15nCE7rVjxw798pe/TN8e6L2Aczg7fv3rX+vSSy8d8D7O4eFpbm7WDTfcoO985zuaPXs278VZ1nd+JXvfhwnPOjTBK1asUDQaVSQS0fr163XxxRen7z/++OPl9/vV3NwsSdq4caPOP/98u4brWjt27NCJJ56oQCDQ774xY8boRz/6kT744AMZhqE1a9Zk/A1gzqmnnqqdO3fq/fffVzKZ1ObNm/udq8XFxTr99NP1i1/8QpK0YcMGzudhiEQi+qd/+ifdeuutmj9//oCPCQQC+ulPf5petefJJ5/kfDbJMAwtW7ZM4XBY8Xhca9eu7Td3nMOj19nZqe7ubk2YMGHA+zmHzfvoo4+0cOFCrVy5UjNmzJDEe3E2DTS/dr8PU9qTdMEFF2j79u267LLLlEqldM011+i0007T17/+dTU0NOiLX/yiVq5cqSVLligSiegLX/jCEXvEcGQffPCBjj322Ixjvef4nnvu0Te/+U3F43HV1NQM2N6Bwfn9fi1fvly33HKLotGoamtrNX36dEnS3XffrQsvvFAXXXSRli5dqrvuukv/8R//oeOOO07/9m//ZvPI3WPdunX65JNP9Oijj+rRRx+VJF144YW69dZbM+b4/vvv1/e//311d3frxBNPTC+vhMFNmjRJN998s66++molEglNnTpVM2fOlMQ5nE27d+/u934siXN4BB5++GFFo1EtX748fWzOnDm8F2fJQPN76aWX2vo+7DEMw8jKTwIAAADyHG0bAAAAgEmEZwAAAMAkwjMAAABgEuEZAAAAMInwDAAAAJhEeAaAPHbXXXfp4YcfHvQx+/fvz1h+c9asWfr888+tHhoAuBLrPANAgQuHw3rzzTfTtzdu3GjjaADA2ag8A4ANXnnlFV111VW69dZbVVdXp6uuukrvvvuu9u/frzvuuEMzZ85UXV2dVqxYoUQiIUn6h3/4B61atUqXX365pk+frl/96leSpJ///OdasGBB+mf3vd1j3bp1uuqqq3TZZZfpggsu0M9+9jNJ0qJFi9Td3a1Zs2YpmUzqlFNOUWdnpyRp9erVuvTSS1VXV6eGhob0lsPz5s3Tv/7rv2ru3Lm68MILdffddyuVSlk6ZwDgBIRnALBJa2ur5s2bp02bNunyyy/Xd7/7Xd17770aN26cNm3apGeeeUY7duzQI488IklKJpM66qij9POf/1z333+/Fi9enA65Qzlw4ICefvpp/ed//qc2bNigVatW6Uc/+pEk6b777tOYMWO0ceNGeb3e9HOeeeYZ/e53v9O6deu0adMmnXzyybrrrrvS9+/atUtPPPGEnn32Wb344ov64x//mMXZAQBnIjwDgE0mTZqk008/XZJ0xRVX6O2339bmzZt17bXXyuPxqKSkRHPmzNGLL76Yfs61116bfm5VVZVeffVVU681duxYPfjgg2pqatL999+vBx98UF1dXYM+58UXX9Tll1+uQCAgSbruuuv08ssvKxaLSZIuuOACFRUVqbS0VBMnTlQ4HB72HACA2xCeAcAmvau8PQzDkMfjSd9OpVLpto2+z0mlUvJ6vfJ4PDIMI308Ho/3+7l/+ctfdNlll+nDDz/UlClTdNtttw05vlQqNehYxowZk/533zEAQL4iPAOATf70pz/pT3/6kyRp7dq1Ou2003TJJZfoySeflGEYisViCoVC+spXvpJ+zoYNGyRJb731lnbu3KkzzjhDwWBQ77zzjqLRqOLxuH75y1/2e63W1lYFg0F961vf0rnnnqvf/OY3kg61gvh8PiWTyX7h97zzztMzzzyTrlA/8cQTOuOMM1RSUmLFdACAK7DaBgDY5G/+5m90//3368MPP1QwGNSKFSs0duxY3Xvvvaqrq1M8Htd5552nb3zjG+nnbNu2TaFQSKlUSqtWrVJZWZnOOeccnXHGGbrkkktUXl6uM888Uzt27Mh4rXPOOUfr1q3T9OnT5fF49OUvf1nBYFDvv/++Jk6cqC996UuaMWOG1qxZk37OlVdeqY8++khXXXWVUqmUJk6cqJUrV+ZsfgDAiTwG37MBQM698sor+sEPfqDNmzebfs4pp5yil156ScFg0MKRAQAGQ9sGAAAAYBKVZwAAAMAkKs8AAACASYRnAAAAwCTCMwAAAGAS4RkAAAAwifAMAAAAmPT/A4PhrztsngSNAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 864x576 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "df.plot(kind='scatter',x='population',y='profit', figsize=(12,8))\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***\n",
    "### 计算代价函数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "首先，我们将创建一个以参数θ为特征函数的代价函数\n",
    "$$J\\left( \\theta  \\right)=\\frac{1}{2m}\\sum\\limits_{i=1}^{m}{{{\\left( {{h}_{\\theta }}\\left( {{x}^{(i)}} \\right)-{{y}^{(i)}} \\right)}^{2}}}$$\n",
    "其中：\\\\[{{h}_{\\theta }}\\left( x \\right)={{\\theta }^{T}}X={{\\theta }_{0}}{{x}_{0}}+{{\\theta }_{1}}{{x}_{1}}+{{\\theta }_{2}}{{x}_{2}}+...+{{\\theta }_{n}}{{x}_{n}}\\\\]  (假设函数)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "#代价函数\n",
    "def computeCost(X,y,theta):\n",
    "    inner=np.power(np.dot(X,theta.T)-y,2)\n",
    "    return np.sum(inner)/(2*len(X))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们已经建立了线性回归的数据。在下面几行中，我们在数据中增加了另一个维度，以适应θ0拦截项。我们还将初始参数初始化为0，学习率alpha初始化为0.01。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2, 1)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "theta=np.zeros((2,1))\n",
    "#为X增加维度\n",
    "df.insert(0,'ONE',1)\n",
    "#取列\n",
    "n=df.shape[1]\n",
    "#更新X\n",
    "X=df.iloc[:,0:n-1]\n",
    "y=df.iloc[:,n-1:n]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "#代价函数是应该是numpy矩阵，所以我们需要转换X和Y，然后才能使用它们。 我们还需要初始化theta。\n",
    "X=np.matrix(X.values)\n",
    "y=np.matrix(y.values)\n",
    "theta=np.matrix(np.array([0,0]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "32.072733877455676"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "computeCost(X,y,theta)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 批量梯度下降 batch gradient decent"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$${{\\theta }_{j}}:={{\\theta }_{j}}-\\alpha \\frac{\\partial }{\\partial {{\\theta }_{j}}}J\\left( \\theta  \\right)$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "ename": "SyntaxError",
     "evalue": "unexpected EOF while parsing (<ipython-input-9-7bc2f6c8d72b>, line 2)",
     "output_type": "error",
     "traceback": [
      "\u001b[1;36m  File \u001b[1;32m\"<ipython-input-9-7bc2f6c8d72b>\"\u001b[1;36m, line \u001b[1;32m2\u001b[0m\n\u001b[1;33m    \u001b[0m\n\u001b[1;37m    ^\u001b[0m\n\u001b[1;31mSyntaxError\u001b[0m\u001b[1;31m:\u001b[0m unexpected EOF while parsing\n"
     ]
    }
   ],
   "source": [
    "def gradientDecent(X,y,theta,alpha,iters):  #alpha是学习率,iters为迭代次数\n",
    "    temp=np.matrix(np.zeros(theta.shape)) #将theta转换为可计算的矩阵\n",
    "    #parameters表示参数\n",
    "    parameters=int(theta.ravel().shape[1]) #theta.ravel()：将多维数组theta降为一维，.shape[1]是统计这个一维数组有多少个元\n",
    "    #初始化代价函数\n",
    "    cost=np.zeros(iters)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
